Conversation
|
Introduce Randomized Batch Sizes in Batch Utilities for Property Testing This PR updates the batch utility logic to split input data into batches of random sizes (rather than consistently sized slices) for testing whether variable batch sizes impact downstream processing. The batching function is modified to use a random size (within the allowed maximum) during the batch split process, aligning with the stated intent to evaluate robustness against non-uniform batches. Key Changes: Affected Areas: This summary was automatically generated by @propel-code-bot |
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
|
Do not merge this as it breaks the client batching lib. |
chromadb/utils/batch_utils.py
Outdated
| offset = 0 | ||
| if len(ids) > max_batch_size: | ||
| while offset < len(ids): | ||
| batch_size = random.randint(1, max_batch_size): |
There was a problem hiding this comment.
[CriticalError]
There's a syntax error in your code - the line with random.randint() has a colon at the end which is invalid Python syntax.
Description of changes
Testing if randomly sized batches break things.
Test plan
CI.
Documentation Changes
N/A